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ABSTRACT 



The National Assessment of Educational Progress (NAEP) 1998 
Writing Assessment measured student writing performance at the 4th, 8th, and 
12th grades. Scoring guides for each grade allowed scorers to objectively 
evaluate students’ work. This issue of NAEPfacts includes a 4th-grade 
narrative scoring guide, along with samples of student work at each of six 
levels of performance on the scoring guide. ( Author /RS) 



Reproductions supplied by EDRS are the best that can be made 
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NAEP Scoring of Fourth-Grade Narrative Writing 



Abstract: The National Assessment of Educational Progress 
(NAEP) 1998 Writing Assessment measured student writing 
performance at the 4th, 8th, and 1 2th grades. Scoring guides 
for each grade allowed scorers to objectively evaluate stu- 
dents ’ work. This issue ofNAEPfacts includes a 4th-grade 
narrative scoring guide, along with samples of student work at 
each of six levels of performance on the scoring guide. 



The NAEP Writing Framework developed by the 
National Assessment Governing Board, set six 
overarching goals for the NAEP 1 998 Writing As- 
sessment: 



• Students should write for a variety of purposes: 
narrative, informative, and persuasive. 

• Students should write on a variety of tasks and 
for many different audiences. 

• Students should write from a variety of stimu- 
lus materials, and within various time con- 
straints. 

• Students should generate, draft, revise, and edit 
ideas and forms of expression in their writing. 

• Students should display effective choices in the 
organization of their writing. They should in- 
clude detail to illustrate and elaborate their 
ideas, and use appropriate conventions of writ- 
ten English. 

• Students should value writing as a communica- 
tive activity. 

- Fourth-graders were given two writing topics, or 
“prompts,” out of a possible 20 and were given 25 
min utes to write on each. Each topic was classified 
according to purpose as either narrative, informa- 
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tive, or persuasive. Three 4th-grade topics have 
been released to the public. 

Scoring guides were developed for each writing 
purpose. The guides established six levels of student 
performance for each writing purpose, ranging from 
“Unsatisfactory” to “Excellent.” This issue of 
NAEP facts includes the complete text of the 4th- 
grade “narrative” scoring guide on page 2, the com- 
plete text of a narrative prompt asking students to 
write a story about a magic castle, examples of stu- 
dent writing at each of the six levels in response to 
the “magic castle” prompt, and a discussion of how 
the scoring guide applies to the six student writing 
samples. 

Focused Holistic Scoring 

The scorers of the NAEP 1 998 writing assessment 
used a scoring method described a s “focused hol is- 
tic scoring.” This approach combines holistic and 
“primary trait” scoring. A strict holistic approach to 
the scoring of writing treats a writing task as a 
“springboard” for writing. A particular writing task 
is given to students as a stimulus to engage them 
and inspire them to write, and students’ responses 
are scored in terms of the overall writing quality. 
The “primary trait” method of scoring writing, on 
the other hand, is concerned with how well students 
respond to a specific topic. For example, if students 
are asked to write about whether they like adventure 
movies, students who do not address the topic of 
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• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



Fourth-Grade Narrative Writing Scoring Guide 



1. Unsatisfactory Response (may be character- 
ized by one or more of the following) 

• Attempts a response, but may only paraphrase 
the prompt or be extremely brief. 

• Exhibits no control over organization. 

• Exhibits no control over sentence formation; 
word choice is inaccurate across the response. 

• Characterized by misspellings, missing words, 
incorrect word order; errors in grammar, spell- 
ing, and mechanics severely impede under- 
standing across the response. 

2. Insufficient Response (may be characterized 
by one or more of the following) 

• Attempts a response, but is no more than a 
fragment or the beginning of a story OR is 
very repetitive. 

• Is very disorganized OR too brief to detect or- 
ganization. 

• Exhibits little control over sentence boundaries 
and sentence formation; word choice is inaccu- 
rate in much of the response. 

• Characterized by misspellings, missing words, 
incorrect word order; errors in grammar, spell- 
ing, and mechanics are severe enough to make 
understanding very difficult in much of the re- 
sponse. 

3. Uneven Response (may be characterized by 
one or more of the following) 

• Attempts to tell a story, but tells only part of a 
story, gives a plan for a story, or is list-like. 

• Lacks a clear progression of events; elements 
may not fit together or be in sequence. 

• Exhibits uneven control over sentence bounda- 
ries and may have some inaccurate word 
choices. 

• Errors in grammar, spelling, and mechanics 
sometimes interfere with understanding. 



4. Sufficient Response 

• Tells a clear story with little development; has few 
details. 

• Events are generally related; may contain brief di- 
gressions or inconsistencies. 

• Generally has simple sentences and simple word 
choice; may exhibit uneven control over sentence 
boundaries. 

• Has sentences that consist mostly of complete, 
clear, distinct thoughts; errors in grammar, spell- 
ing, and mechanics generally do not interfere with 
understanding. 

5. Skillful Response 

• Tells a clear story with some development, includ- 
ing some relevant descriptive details. 

• Events are connected in much of the response; 
may lack some transitions. 

• Exhibits some variety in sentence structure and 
exhibits some specific word choices. 

• Generally exhibits control over sentence bounda- 
ries; errors in grammar, spelling, and mechanics 
do not interfere with understanding. 

6. Excellent Response 

• Tells a well-developed story with relevant de- 
scriptive details across the response. 

• Events are well connected and tie the story to- 

gether with transitions across the response. 

• Sustains varied sentence structure and exhibits 
specific word choices. 

• Exhibits control over sentence boundaries;, errors 
in grammar, spelling, and mechanics do not inter- 
fere with understanding. 



adventure movies will receive lower scores than 
those who do. 

The “focused holistic scoring approach” used by 
NAEP, as with all holistic approaches, requires 
scorers to rate the overall quality of the writing, re- 
gardless of how students choose to respond to spe- 
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cific aspects of a given topic. In contrast to some 
holistic approaches to writing scoring that offer 
very general guidelines, NAEP scorers were given 
detailed scoring guides that focused their attention 
on specific characteristics of students’ writing (or- 
ganization, development, syntax, and mechanics). 
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In this sense, the “traits” of writing now at issue for 
NAEP scoring of writing have shifted from a con- 
cern with topic-related traits of student responses to 
traits associated with overall quality of writing. 

The basic assumptions of the NAEP focused holis- 
tic scoring approach are: 

• Each of the factors involved in writing is re- 
lated to all others and that no one factor can be 
separated from the others. 

• A writer is entitled to make some mistakes, 
given the 25-minute time limit, the lack of re- 
course to a dictionary, and the lack of time for 
reviewing and editing. 

• Scorers should read each response as a whole — 
without focusing on each mistake (but still be- 
ing aware of them) — to judge the level of writ- 
ing ability demonstrated by the student. 

• After thorough training on the scoring of re- 
sponses written on a given task, scorers should 
quickly read an entire response and assign a 
score based on the total impression conveyed 
by the response. 

• Scorers should ignore their personal standards 
of what constitutes good writing and embrace 
the criteria of the scoring guide. 

• Scorers should read supportively rather than 
critically. 

Narrative Writing 

Narrative writing involves the production of stories 
or personal essays. Practice with these forms helps 
writers to develop a facility for spontaneous and 
colloquial language. Also, informative and persua- 
sive writing can benefit from many of the strategies 
used in narrative writing. For example, there must 
be an effective ordering of events when relating an 
incident as part of a report. 

Sometimes narrative writing contributes to an 
awareness of the world as the writer creates, ma- 
nipulates, and interprets reality. Such writing — 
whether fact or fiction, poem, play, or personal es- 
say — requires close observation of people, objects, 
and places. Further, this type of writing fosters crea- 
tivity, imagination, and speculation by allowing the 



writer to express thoughts and then stand back, as a 
more detached observer might, and grasp more fully 
what is being felt and why. Thus, narrative writing 
offers a special opportunity to analyze and under- 
stand emotions and actions. 

Fourth-grade students were given a number of nar- 
rative topics on the 1998 Writing Assessment. One 
of these topics asked students to write about a child 
encountering a castle that appears overnight as if by 
magic. The complete text of this topic is given be- 
low: 

Castle 

One morning a child looks out the window and 
discovers that a huge castle has appeared over- 
night. The child rushes outside to the castle and 
hears strange sounds coming from it. Someone is 
living in the castle! 

The castle door creaks open. The child goes in. 

Write a story about whom the child meets and what 
happens inside the castle. 

In the imaginative stories written for this topic, 
“Castle,” characters sometimes appear and disap- 
pear rather suddenly. Students who received ratings 
in the upper half of the six levels on the scoring 
guide (“Sufficient” or better) were able to weave 
coherent stories, making effective use of suspense 
and surprise. 

1. Sample “Unsatisfactory” Response 

Student response: The child meet a 
castle and go in the castle^ 

The “Unsatisfactory” rating was given to 2 percent 
of the responses to this topic. As the scoring guide 
on page 2 indicates, responses at this level tended 
either to be so brief that they did not develop a story 
at all, or to be hard to understand throughout. In the 
response shown, the student only paraphrases the 
topic. 

2. Sample “Insufficient” Response 

Student response: One morning a child 
looks out a window and sees that a 
castle appears overnight. She runs to 
the castle and inside she meet a gi- 
ant. The giant let her in. He asked 
her her name. He was married to an- 
other giant. 

The “Insufficient” rating was given to 12 percent of 



the responses to this topic. In “Insufficient” re- 
sponses, students produced only the beginning of a 
story, wrote very disorganized stories, or wrote re- 
sponses that were understandable only in part. In 
the response shown, the student begins to tell a 
story, introducing a new character, the “giant,” but 
does not progress beyond that point. 

3. Sample “Uneven” Response 

Student response: He saw died fish 
when he walked in the door. Than he 
herd something a bat fly and turn in 
to a vampire. He ran out of the door 
yelling for help while running out the 
door. Then the" vampire turned in to a_ 
bat again. And the boy never went 
there again. Then a year later a girl 
went there and opened the door and 
seen died fish at the “door and seen a 
bat flying and than therd in to a vam- 
pire and yelled out the door yelling 
for help. And than the vampire turned 
in to a bat again. And it gos on on on 
on on on and on on on again. 

The “Uneven” rating was given to 31 percent of the 
responses to this topic. In such responses, students 
attempted to tell an entire story, but the attempt was 
incomplete or disorganized. In the “Uneven” re- 
sponse shown, there is some dramatic action (“than 
he herd something a bat fly and turn in to a vam- 
pire”). That action, however, is repetitive, as the 
events are not connected to form a coherent story: 
“And then the vampire turned in to a bat again. And 
it gos on on on on on on and on on on again.” 

4. Sample “Sufficient” Response 
Student response: One day a 13 year 
old boy woke up and found beautiful 
castle with a purple, pink, red, blue, 
orange and yellow rainbow. He decided 
to get up. Then he got dressed and 
went to see what was in the castle. He 
walked to the door and knocked nobody ' 
answered so he knocked again still no 
answer . 

Then the boy went in. It was pretty 
dark inside not like the outside of 
the color: He looked around and saw 
that there must be someone living 



here. It was very clean he could see 
that it was clean in the dark. 

He went up stairs to a room and opened 
it and there he saw the ugly monster, 
and behind him was a pretty princess . 
He was in love. The princess was tide 
up. 

The “Sufficient” rating was given to 38 percent of 
the responses to this topic. In such responses, stu- 
dents told complete stories that were organized and 
clear, but lacking in detail. In the “Sufficient” re- 
sponse, the student provides a clear but bare plot. 

He or she includes the vivid detail of the colors of 
the rainbow in the first sentence, but uses detail 
sparingly beyond that. Though the story does not 
conclude, enough action occurs that most of the de- 
velopment is clear. The simple, but essentially clear 
and correct, sentence structure and vocabulary are 
typical of responses at this level. 

5. Sample “Skillful” Response _ 

Student response: First He sees a dark 
room filled with object some big and 
some small some short some tall. As he 
is reaching for the light switch some- 
thing grabs his hand and turns it on 
for him. As it turns on the boy sees a 
woman so beautiful she captures his 
eyes. They look so much alike. They 
walk through the castle telling each 
other stories about each other showing 
each other pictures from there life. 
The girl says she had a long lost 
brother from long ago. She says he 
looks like this and showed him the 
picture. The boy says he has a picture 
just like that of himself Then the 
girl realizes the boy in the picture 
is her long Lost Brother. 

The “Skillful” rating was given to 14 percent of the 
responses to this topic. In such responses, students 
used details to develop their stories in parts of the 
response. They provided a good structure to their 
stories, though with occasional lack of transitions. 

In the sample “Skillful” response, the plot occasion- 
ally shifts abruptly, as when the boy “sees a 
woman” who looks like him and they suddenly start 
to “walk through the castle.” Though the ending is 



concise, the student ties up the story with the revela- 
tion “Then the girl realizes the boy in the picture is 
her long Lost Brother.” 

6. Sample “Excellent” Response 
Student response: "Wow a castle!" said 
John. He had know clue of how it got 
here or where it came from? He walked 
inside and found that it was rather 
damp. He wandered around until finally 
he saw someone. This person didn't 
look normal. He was dressed in royalty 
with a purple cape and a crown of jew- 
els. Then the person spoke out. "There 
you are you' re supposed to be training 
right now." John had know clue what he 
was talking about. Suddenly he thought 
of something, was this the King of the 
castle? He finally got the nerve to 
ask a question. He asked "Who are 
you." He answered "I'm the King." John 
was shocked. Then the King told him to 
get on his armor. John thought and 
thought. Then he knew what he was 
talking about. He thought he was a 
knight. John thought again. If he was 
to be a knight then he would never see 
his family again. Then he thought of 
his older sister, Jennifer. He decided 
to be a knight. After about 2 months 
he— finally was knighted. He fought 
many dragons and man. He finally died 
but is still a legend today. 

The End 
By: Unknown 

The “Excellent” rating was given to 3 percent of the 
responses to this topic. Such responses may have 
excelled through good development of plot, charac- 
ters, or dialogue. In the response shown, the student 
uses dialogue effectively, develops characters, and 
provides a coherent plot. The student shows good 
control of language for a 4 th -grader and includes . 
vivid details about appearance — “He was dressed in 
royalty with a purple cape and a crown of jewels.” 

Conclusion 

The scoring guides used in the NAEP 1998 Writing 
Assessment set six possible levels of writing per- 
formance for students, from “Unsatisfactory” to 
“Excellent.” Among 4 th -graders who wrote on the 



“Castle” narrative writing topic, 2 percent were 
rated “Unsatisfactory,” 1 2 percent were rated 
“Insufficient,” 31 percent were rated “Uneven,” 38 
percent were rated “Sufficient,” 14 percent were 
‘rated “Skillful,” and 3 percent were rated “Excel- 
lent.” 

Scoring guides, or “rubrics,” are a widely used 
means of ensuring objective scoring of student work 
that requires a judgement of quality. Teachers using 
scoring guides in the classroom can use the guides 
not only to evaluate student work but also to explain 
to students where their work needs improvement. 3 

Endnotes 

1 pp. 5-1 1, National Assessment Governing Board. Writing frame- 
work and specifications for the 1998 National Assessment of Educa- 
tional Progress. Washington, DC: Author. Web address: 

h ttp.V/www. nagb. org/pu bs/writing.pdf 

2 The three topics are available in The NAEP 1998 Writing Report 
Card. Additional information is available from the “Sample Ques- 
tions” section of the NAEP Web Site: 
http://nces.ed.gov/nationsreportcard/ITMRLSAntro.shtml 

3 For more information on the use of scoring guides or rubrics, see 
Moskal, Barbara M. Scoring Rubrics: What, When and How? Practi- 
cal Assessment, Research & Evaluation , 7(3). 
http://ericae.net/pare/getvn.asp?v= 7&n~3 

The ERIC Clearinghouse has a discussion of rubrics, a bibliography, 
and additional links at 

http://ericae.net/faqs/rubrics/scoring_ru brics. htm 

For Further Information 

The NAEP 1998 Writing Report Card, NCES 1999-486, 
is the complete report. Single copies are available free 
from ED Pubs, P.O. Box 1398, Jessup, MD 20794-1398. 
Copies may also be.obtained over the. World Wide Web: 
http://nces.ed.gov/nationsreportcard/writing/ 

Two additional NAEPfacts, Scoring of Eighth-Grade 
Informative Writing (NCES 2000-506), and Scoring of 
Twelfth-Grade Persuasive Writing (NCES 2000—488), 
are also available. Single copies are available free from 
ED Pubs, P.O. Box 1398, Jessup, MD 20794-1398. 
Copies may also be obtained over the World Wide Web: 
http://nces.ed.gov/nationsreportcard/writing/ 

NAEPfacts briefly summarize findings from the National 
Assessment of Educational Progress (NAEP). The series 
is a product of the National Center for Education Statis- 
tics, Gary Phillips, Acting Commissioner, and Peggy 
Carr, Associate Commissioner for Education Assess- 
ment. This issue of NAEPfacts was prepared by Sheida 
White of NCES and Alan Vanneman of the Education 
Statistics Services Institute, based on previously pub- 
lished material. 
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To order NAEP publications, call toll free 1-877-4ED- 
Pubs (1-877-433-7827), TTY/TDD 1-877-576-7734; 
E-mail: edpubs@inet.ed.gov 
Internet: http://www.ed.gov/pubs/edpubs.html 

The NCES World Wide Web Home Page is: 
http://nces. edgov/ 

The NAEP World Wide Web Home Page is: 
http.V/nces.edgov/nationsreportcard/ 
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